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METHOD OF FAST AUTOMATIC EXPOSURE OR GAIN CONTROL IN A MOS 

IMAGE SENSOR 

Technical Field of the Invention 

The present invention relates generally to CMOS image sensors, and more particularly, to 
a method of controlling an exposure time and/or gain of an image sensor. 

Background of the Invention 

Integrated circuit technology has revolutionized various fields including computers, 
control systems, telecommunications, and imaging. For example, in the imaging field, CMOS 
image sensors have proved to be less expensive to manufacture relative to CCD imaging devices. 
Further, for certain applications CMOS devices are superior in performance. The pixel elements 
in a MOS device can be made smaller and therefore provide a higher resolution than CCD image 
sensors. In addition, the signal processing logic necessary can be integrated alongside the 
imaging circuitry, thus allowing for a single integrated chip to form a complete stand alone 
imaging device. CMOS image sensors are now available from a variety of manufacturers, 
including the assignee of the present invention, OmniVision Technologies, Inc. 

The primary building block of an image formed by a CMOS image sensor is a pixel. The 
number, size and spacing of the pixels determine the resolution of the image generated by the 
imaging device. The pixels of a CMOS image sensor are semiconductor devices that transform 
incident light photons into current signals. The signal produced by each pixel is generally 
extremely small. 

One important parameter that the CMOS image sensor must be able to control is the 
exposure time of each pixel to incident light. Similar to light exposure time for photographic 
film, the exposure time of each pixel must be adjusted to compensate for variations in lighting 

1 



Utility Patent Attorney Docket No. : 004320.P054 

conditions, such as for indoor or outdoor lighting. An exposure time that is too long will result 
in an image that is overly bright and washed out. In contrast, an exposure time that is too short 
will result in an image that is dark and difficult to view. 

Another method of varying the intensity of the pixel signals is to adjust the gain of the 
amplification circuitry. In many ways, controlling the gain is substantially similar to controlling 
the exposure time. By varying one or both, the image generated by the image sensor can be 
optimized for viewing. 

U.S. Patent No. 5,734,426 describes one prior art method of controlling exposure time. 
While adequate for many applications, the speed at which the exposure time can be 
automatically changed is insufficient for certain applications. For example, for automobile 
applications, where the automobile may travel from sunlight, to darkness (such as in a tunnel), 
back to sunlight, it is important for the image sensor to be able to adjust its exposure time 
quickly. 
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Brief Description of the Drawings 
In the drawings, like reference numerals refer to like parts throughout the various views 
of the non-limiting and non-exhaustive embodiments of the present invention, and wherein: 
Figure 1 is a schematic diagram of a CMOS image sensor. 
Figure 2 is a flow diagram illustrating the method of the present invention. 
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Detailed Description of the Dlustrated Embodiments 
In the following description, numerous specific details are provided, such as the 
identification of various system components, to provide a thorough understanding of 
embodiments of the invention. One skilled in the art will recognize, however, that the invention 
can be practiced without one or more of the specific details, or with other methods, components, 
materials, etc. In still other instances, well-known structures, materials, or operations are not 
shown or described in detail to avoid obscuring aspects of various embodiments of the invention. 

Reference throughout this specification to "one embodiment" or "an embodiment" means 
that a particular feature, structure, or characteristic described in connection with the embodiment 
is included in at least one embodiment of the present invention. Thus, the appearance of the 
phrases "in one embodiment" or "in an embodiment" in various places throughout this 
specification are not necessarily all referring to the same embodiment. Furthermore, the 
particular features, structures, or characteristics may be combined in any suitable manner in one 
or more embodiments. 

With reference to FIGURE 1, an architecture for a CMOS imaging array 101 includes a 
rectangular matrix of pixels 103. The number of pixels in the horizontal or x-direction, and the 
number of pixels in the vertical or y-direction, constitutes the resolution of the imaging 
array 101. Each of the pixels 103 in a vertical column routes its signal to a single charge 
amplifier 105. 

The retrieval of information from the pixels 103 follows the well-known raster scanning 
technique. In particular, a row of pixels 103 is scanned sequentially from left to right. The next 
row is then scanned in this manner until all rows have been scanned sequentially from top to 
bottom. At the end of each complete scan of the entire array 101, a vertical blanking period of 
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predetermined time occurs until the raster scanning pattern is repeated. This type of scanning 
follows the NTSC scanning scheme. However, for other applications, different readout protocols 
may be followed. Control circuitry of conventional design is operative to sequentially read the 
pixels 103 in this manner. 

As each pixel is scanned, the signal from that pixel is provided to the charge 
amplifier 105 for that column. Thus, the charge amplifiers 105 receive signals sequentially. The 
sequential signals from the charge amplifiers 105 are then forwarded to a second-stage 
amplifier 107, which amplifies the signals so that they may be further processed. In various 
embodiments, either the column amplifier or the second-stage amplifier may be eliminated. The 
rjjo gain of the column amplifier or the second-stage amplifier may be selectively increased or 
fl decreased as the need arises. The readout of the pixels 103, the amplifiers 105 and 107, and the 

3 calculations and processes described below may be carried out by processor 111. 

C 

P As noted above, one important function of the image sensor is to be able to automatically 

i control the gain and exposure time of the image sensor to compensate for varying light 
|5 conditions. Previous attempts to compensate for varying light conditions involve the simple 
D monitoring of pixel output signal magnitudes, and if below a threshold, increase the exposure 
time or gain, and if above a threshold, decrease the exposure time or gain. 

In accordance with the present invention, the amount of decrease or increase in exposure 
time/gain varies depending upon various factors. Specifically, most image sensors are what are 
20 referred to as "line-by-line" exposure devices. In these types of image sensors, each row of the 
image sensor begins the exposure cycle (also referred to as an integration cycle) at a different 
instant in time. As an example, referring to Figure 1, the first row 109-1 of the imaging array 
101 would begin exposure to incident light (typically after a reset operation). A short period 
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later, the second row 109-2 of the imaging array 101 would begin exposure to incident light, also 
after a reset operation. The process continues until all of the rows have been exposed to incident 
light. Importantly, it should be noted that the exposure time of the first row 109-1 would 
continue for a length of time, and would typically not finish until many subsequent rows have 
5 started the exposure operation. 

Indeed, the exposure time for a row is often measured as a number of rows that start the 
exposure operation. For example, the exposure time for a row may be measured as 16 rows or 
20 rows. In other words, the total exposure time for any row is the time it takes to initiate 
exposure of 16 or 20 subsequent rows. 
IS) hi one specific numerical example, assume that the imaging array has 640 columns and 

iS 480 rows (VGA format). At 30 frames per second, one frame must be captured about every 

in 

j <3 0.033 seconds. Still further, each row of the imaging array 101 must start the exposure operation 
"P about every 68 microseconds (0.033 seconds/480 rows). However, 68 microseconds is rarely 
H enough exposure time to capture enough incident light. Indeed, it may take on the order of a 
! B5 millisecond exposure time in order to generate a usable output signal. Thus, from the time of 
f | initiation of exposure to the completion of exposure, 16 or more rows of the imaging array 101 

have started the exposure process. 

As noted above, the measure of the amount of exposure time is done not so much with a 

timer, but rather a counter that counts the number of rows that have started the exposure process. 
20 The term "exposure row count" is used herein to delineate the exposure time, measured by the 

number of rows. As such, if the exposure time is to be modified, the exposure row count must be 

modified. 
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If the exposure row count is nominally set at 16 rows for a typical light intensity, and if 
the imaging array 101 is then moved to a relatively brighter environment, the exposure row count 
should be decreased to decrease the exposure time. However, the decrease in exposure row 
count can only be done in "whole rows", e.g., the exposure row count cannot go from 16 rows to 
5 15.34 rows, only to 1 5 rows. 

Further, the methods and algorithms used to determine whether or not the exposure time 
or gain should be adjusted are many and varied. For example, U.S. Patent No. 5,734,426 to 
Dong and assigned to the assignee of the present invention shows one method of determining 
adjustments to exposure or gain. As another example, our co-pending U.S. patent application 

S Serial No. 09/ , filed November 7, 2001 entitled "METHOD OF FAST AUTOMATIC 

\M EXPOSURE OR GAIN CONTROL IN A MOS IMAGE SENSOR", incorporated herein by 
reference and assigned to the same assignee as the present invention, discloses another 
technique. In any event, various methods are possible for determining when to adjust exposure 
i 2 time or gain. In general, these methods attempt to maintain the average magnitude of the output 
jjs signals from the pixels within a relatively narrow range, e.g. between V H r and Vlo- The narrow 
Fti range is also referred to as a "stable range", where no further adjustments to exposure time or 
gain is required. 

The change from a 16 exposure row count to a 15 exposure row count results in a 
decrease in exposure time of about 6.7%. In many instances, this relatively large amount of 
20 percentage change causes an "overshoot" in the frame exposure time or gain. This overshoot, 
when used in conjunction with an automatic exposure control algorithms, will cause the next 
image to be outside of the stable range. Subsequent adjustments that tune the exposure time or 
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gain may then cause an overshoot in the other direction. This process will result in flicker of the 
image. 

The present invention combines adjustment of exposure time and gain control to provide 
a smooth transition in the image from varying light conditions. The method of the present 
invention is shown in Figure 2. In general, in accordance with the present invention, the 
exposure row count will be incremented or decremented by an integer value if the change in 
exposure time is not greater than the magnitude of the stable range. However, if the change in 
exposure time due to an increment or decrement of one row in the exposure row count is greater 
than the stable range, the gain control of the image sensor is modified to limit the change in 
exposure time. 

As an example, in one embodiment, the stable range is about 1/16 (or about 6.25%) of the 
total signal range. It should be noted that the stable range for a particular image sensor will vary 
based upon the application and specific type of image sensor. The stable range magnitude of 
1/16 of the total signal range is merely exemplar. Thus, if the signal range is 0 to 1.0, then the 
stable range would be between 0.46875 and 0.53125. The magnitude of the stable range is thus 
0.0625. Assume that the exposure row count is 20, and that it is determined that the exposure 
time should be reduced to an exposure row count of 19. This results in a 5% decrease (1/20) in 
the exposure time. This percentage decrease in the exposure time is less than the magnitude of 
the stable range (6.25%). Therefore, in this situation, the change in the exposure time would not 
under any circumstance cause the image to fall outside of the stable range, thereby preventing the 
overshoot and flicker problem. 

However, in a second example, assume that the exposure row count is 15, and that it is 
determined that the exposure time should be reduced to an exposure row count of 14. This 
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results in a 6.667% decrease (1/15) in the exposure time. This percentage decrease in the 
exposure time is greater than the magnitude of the stable range (6.25%). Therefore, in this 
situation, the change in the exposure time would, in some circumstances, cause the image to fall 
outside of the stable range, thereby raising the possibility of the overshoot and flicker problem. 
5 As seen in Figure 2, the method of the present invention begins at box 201 by capturing a 

first image. At box 203, the first image is analyzed to determine if the image falls within the 
stable range. As noted above, this process may follow the methods disclosed in U.S. Patent No. 
5,734,426 or our co-pending patent application identified above. After the analysis is complete, 
a determination is made as to whether the exposure time needs to be increased or decreased. If 
i;tf) not, the image sensor continues to capture images at box 201 . 

m If, however, the exposure time needs to be adjusted, at box 205, the exposure row count 

l: 3 is adjusted appropriately, typically a one row decrement or a one row increment. Next, at box 

[1 SET. 

207, a determination is made as to whether the resultant adjustment in the exposure row count 
]t would change the exposure time (on a percentage basis) by more than a threshold amount. In 
!. lb one embodiment, the threshold amount is approximately the magnitude of the stable range, 
f U If the resultant adjustment does not change the exposure time by more than the threshold 

amount, the next image is captured at box 201. However, if the resultant adjustment does change 
the exposure time by more than the threshold, an adjustment to the amplifier gain is made. Thus, 
the exposure time and the gain control is adjusted to ensure a smooth adjustment of the image 
20 brightness. 

In one embodiment, when the exposure row count is decreased, the amount of gain is 
increased by an amount to bring the total effective exposure and gain to some measure, between 
the previous exposure row count and the decremented exposure row count. Similarly, when the 
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exposure row count is to be increased, the amount of gain is decreased by an amount to bring the 
total effective exposure and gain to some measure between the previous exposure row count and 
the incremented exposure row count. 

In one embodiment, when the exposure row count is decreased, the gain is increased by 
5 some percentage dependent upon the current exposure row count. For example, if the exposure 
row count is between 9 and 16, the gain will be increased by a maximum of 6.25%. If the 
exposure row count is between 5 and 8, the gain will be increased by a maximum of 12.5%. If 
the exposure row count is 3 or 4, the gain will be increased by a maximum of 25%. If the 
exposure row count is 2, the gain will be increased by a maximum of 50%. The foregoing is but 

!ji one example of how the gain control can be adjusted to increase the smoothness by which image 

in brightness is automatically adjusted. 

'"3 Further, it can be appreciated that the actual calculations and determinations may take 

::j== place within and controlled by processor 111. 

H While the invention is described and illustrated here in the context of a limited number of 

! |5 embodiments, the invention may be embodied in many forms without departing from the spirit of 
E the essential characteristics of the invention. The illustrated and described embodiments are 
therefore to be considered in all respects as illustrative and not restrictive. Thus, the scope of the 
invention is indicated by the appended claims rather than by the foregoing description, and all 
changes which come within the meaning and range of equivalency of the claims are intended to 
20 be embraced therein. 



